Telepresence systems allow a user at one location to view a remote location (e.g., a conference room) as if they were present at the remote location. Mutually-immersive telepresence system environments allow the user to interact with individuals present at the remote location. In a mutually-immersive environment, the user occupies a display booth, which includes a projection surface that typically surrounds the user. Cameras are positioned about the display booth to collect images of the user while other cameras are positioned at the remote location to collect images of the remote location. Live color images of the user are transmitted to the remote location, concurrent with projection of live video on the projection surface surrounding the user and reproduction of sounds from the remote location. A surrogate at the remote location shows images of the user and reproduces the user's voice at the remote location.
Projecting video images on multiple sides of the user or surrounding the user is important for providing the user a strong sense of immersion in the remote location. In addition, it allows the user to look all around the remote location without having to use tele-operated motion of remote cameras, which is highly unimmersive. Similarly, it is important to display multiple sides of the user at the remote location so that people on all sides of the surrogate can tell what the user is looking at.
Transmission of multiple streams of high quality video between the remote location and the location of the user would consume significant bandwidth. For example, even with state-of-the-art video compression techniques, over 20 Mb/s may be required to present the remote location at a visual acuity permitting 20/100 vision over a 360 degree horizontal field of view at 15 frames-per-second (fps). This bandwidth requirement can preclude the use of such telepresence systems in certain circumstances, such as where communications occur over wireless networks or transoceanic cables.
Therefore, what is needed is a technique for reducing the bandwidth required in transmitting video images. What is further needed is such a technique for a telepresence system in which an immersive environment is maintained.